Papers with healthcare setting
Enhancing Healthcare LLM Trust with Atypical Presentations Recalibration (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for eliciting and calibrating large language models have focused on general reasoning datasets, yielding only modest improvements. |
| Approach: | They propose a method which leverages atypical presentations to adjust model confidence estimates. |
| Outcome: | The proposed method reduces calibration errors by approximately 60% on three medical question answering datasets and outperforms existing methods such as vanilla verbalized confidence, CoT verbalised confidence and others. |
Not What the Doctor Ordered: Surveying LLM-based De-identification and Quantifying Clinical Information Loss (2025.emnlp-main)
Copied to clipboard
Kiana Aghakasiri, Noopur Zambare, JoAnn Thai, Carrie Ye, Mayur Mehta, J Ross Mitchell, Mohamed Abdalla
| Challenge: | De-identification is an application of NLP where automated algorithms remove identifying information of patients and providers. |
| Approach: | They propose to use generative large language models to de-identify patients and providers . they propose to validate existing metrics to quantify extent of inappropriate removal . |
| Outcome: | The proposed method is based on a survey of LLM-based de-identification research . it shows that the models perform poorly in identifying clinically relevant changes . |